INVERTIBILITY OF RANDOM MATRICES: NORM OF 

THE INVERSE 



MARK RUDELSON 

Abstract. Let A be an n x n matrix, whose entries are indepen- 
dent copies of a centered random variable satisfying the subgaus- 
sian tail estimate. We prove that the operator norm of does 
not exceed Cn^/"^ with probability close to 1. 



1. Introduction. 

Let A be an n X n matrix, whose entries are independent identically 
distributed random variables. The spectral properties of such matrices, 
in particular invertibility, have been extensively studied (see, e.g. the 
survey |DSj ). While A is almost surely invertible whenever its entries 
are absolutely continuous, the case of discrete entries is highly non- 
trivial. Even in the case, when the entries of A are independent random 
variables taking values ±1 with probability 1/2, the precise order of 
probability that A is degenerate is unknown. Komlos |KH IK2 j proved 
that this probability is o(l) as n — > oo. This result was improved by 
Kahn, Komlos ans Szemeredi j KKSj , who showed that this probability 
is bounded above by 9^ for some absolute constant 9 < 1. The value 
of 9 has been recently improved in a series of papers by Tao and Vu 
TVTHTV2] to ^ = 3/4 + o(l) (the conjectured value is ^ = l/2 + o(l)). 



However, these papers do not address the quantitative characteriza- 
tion of invertibility, namely the norm of the inverse matrix, considered 
as an operator from to W^. Random matrices are one of the stan- 
dard tools in geometric functional analysis. They are used, in particu- 
lar, to estimate the Banach-Mazur distance between finite-dimensional 
Banach spaces and to construct sections of convex bodies possessing 
certain properties. In all these questions the distortion • 
plays the crucial role. Since the norm of A is usually highly concen- 
trated, the distortion is determined by the norm of A~^. The estimate 
of the norm of A~^ is known only in the case when A is a matrix with 
independent A^(0, 1) Gaussian entries. In this case Szarek |Sz2j proved 
that ll^^^ll < c^/n with probability close to 1 (see also |Szlj where 
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the spectral properties of a Gaussian matrix are applied to an impor- 
tant question from geometry of Banach spaces). For other random 
matrices, including a random ±1 matrix, even a polynomial bound was 
unknown. Proving such polynomial estimate is the main aim of this 
paper. 

More results are known about rectangular random matrices. Let F 
be an X n matrix, whose entries are independent random variables. 
If > n, then such matrix can be considered as a linear operator F : 
M" — i> where Y = FM". If we consider a family F„ of such matrices 
with n/N = a for a fixed constant a > 1, then the norms of (F„|y)~-^ 
converge a.s. to (1 — -\/a)^^n^^/^, provided that the fourth moments 
of the entries are uniformly bounded |BYj . The random matrices for 
which n/N = 1 — o{l) are considered in |LPRTj . If the entries of such 
matrix satisfy certain moment conditions and n/N > 1 — c/ logn, then 
||(F|y)~^|| < C{n/N) ■ n~^/^ with probability exponentially close to 1. 

The proof of the last result is based on the e-net argument. To 
describe it we have to introduce some notation. For p > 1 let 
denote the unit ball of the Banach space l^. Let E C M" and let 
i? C be a convex symmetric body. Let e > 0. We say that a set 
F C M" is an e-net for E with respect to B if 

E C [j{x + eB). 

The smallest cardinality of an e-net will be denoted by N{E, B, e). For 
a point X G M", stands for the standard Euclidean norm, and for 
a linear operator T : M" M*^, ||T|| denotes the operator norm of 

Let E C S^~^ be a set such that for any fixed x & E there is a good 
bound for the probability that ||Fx|| is small. We shall call such bound 
the small ball probability estimate. If N{E, B2, e) is small, this bound 
implies that with high probability ||Fx|| is large for all x from an e-net 
for E. Then the approximation is used to derive that in this case ||Fa;|| 
is large for all x & E. Finally, the sphere S^~^ is partitioned in two sets 
for which the above method works. This argument is possible because 
the small ball probability is controlled by a function of A^, while the 
size of an e-net depends on n < A^. 

The case of a square random matrix is more delicate. Indeed, in 
this case the small ball probability estimate is too weak to produce a 
non-trivial estimate for the probability that ||Fa:|| is large for all points 
of an e-net. To overcome this difficulty, we use the e-net argument for 
one part of the sphere and work with conditional probability on the 
other part. Also, we will need more elaborate small ball probability 
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estimates, than those employed in |LPRTj . To obtain such estimates we 
use the method of Halasz, which hes in the foundation of the arguments 
of HKB], |TVTj . jTV2l . 

Let P (Q) denote the probabihty of the event Q, and let denote 
the expectation of the random variable ^. A random variable (3 is called 
subgaussian if for any t > 

(1.1) F{\(3\>t) <Cexp{-ct^). 

The class of subgaussian variables includes many natural types of ran- 
dom variables, in particular, normal and bounded ones. It is well- 
known that the tail decay condition (jl.lj) is equivalent to the moment 

condition (E|/3|p)^^^ < C'^ for all p > 1. 

The letters c, C, C etc. denote unimportant absolute constants, 
whose value may change from line to line. Besides these constants, 
the paper contains many absolute constants which are used through- 
out the proof. For reader's convenience we use a standard notation for 
such important absolute constants. Namely, if a constant appears in 
the formulation of Lemma or Theorem x.y, we denote it Cx.y or c^.y 

The main result of this paper is the following polynomial bound for 
the norm of A~^. 

Theorem 1.1. Let (3 he a centered subgaussian random variable of 
variance 1. Let A he an n ^ n matrix whose entries are independent 
copies of p. Then for any e > qiTi / y/n 

^(^^xeW\\\Ax\\<-^-^'^<e 

if n is large enough. 

More precisely, we prove that the probability above is bounded by 
e/2 + 4exp(— cn) for all n G N. 

The inequality of Theorem 11.11 means that ||^^^|| < Qm • n^^'^/e 
with probability greater than 1 — e. Equivalently, the smallest singular 
number of A is at least g/(Qrn- n^/^). 

An important feature of Theorem 11.11 is its universality. Namely, 
the probability estimate holds for all subgaussian random variables, 
regardless of their nature. Moreover, the only place, where we use the 
assumption that /3 is subgaussian, is Lemma 1221 below. 

2. Preliminary results. 

Assume that / balls are randomly placed in k urns. Let V & {1, . . . , fc}' 
be a random vector whose i-th coordinate is the number of balls con- 
tained in the i-th urn. The distribution of V, called random allocation. 
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has been extensively studied, and many deep results are available (see 
|KSCj ). We need only a simple combinatorial lemma. 

Lemma 2.1. Let k < I and let X{1), . . . ,X{1) be i.i.d. random vari- 
ables uniformly distributed on the set {1, . . . ,k}. Let t] < 1/2. Then 
with probability greater than 1 — rj'' there exists a set J C {1, . . . ,/} 
containing at least 1/2 elements such that 

(2.1) Ei{^'^^i^(^') = ^}i'^^(^)i- 

i=l 

Remark 2.2. The proof yields C{r]) = t]^^^. This estimate is by no 
means exact. 

Proof. Let X = (X(l), . . . , X{1)). Foii = l,...,k denote 

P,{X) = \{j\X{j) = z}\. 
Let 2 < a < k/2 he a. number to be chosen later. Denote 

J(X) = {z|P,(X)>«^}. 

For any X we have E!=i^i(^) = so |/(X)| < k/a. Set J(X) = 
{j I X(j) e /(X)}. Assume that |J(X)| < 1/2. Then for the set 
J'(X) = {1, ...,/} \ J(X) we have | J'(X)| > 1/2 and 

\{j e j\x) I x(j) = = J2 ^ ^ ■ ^ 

Now we have to estimate the probability that |{J(X)}| > 1/2. To this 
end we estimate the probability that J(X) = J and /(X) = / for fixed 
subsets J C {1, . . . , /} and / C {1, . . . , fc} and sum over all relevant 
choices of J and /. We have 

P(|J(X)|>//2)< Yl E ^{J{X) = J,I{X) = I) 

l^l>V2 \I\<k/a 
|J|>«/2 |/|<fc/a 

< 2\k/a) . (^^J • {l/ar 

< A; - (ea)'^/"- (4/a)'/^ 

since the random variables X(l), . . . ,X(/) are independent. If A; < Z 
and a > 100, the last expression does not exceed a"'/*. To complete 
the proof, set a = rj"^. li rj > (2/A;)^/^, then the assumption a < k/2 
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is satisfied. Otherwise, we can set Cirf) = o? > {k/2Y, for which the 
inequahty (|2.1|) becomes trivial 

□ 

The following result is a standard large deviation estimate (see e.g. 
|DSj or |LPRTj . where a more general result is proved). 



Lemma 2.3. Let A = (aij) be an n x n matrix whose entries are i.i.d 
suhgaussian random variables. Then 

P(P : ^ B^W > qi^) < exp(-n). 

We will also need the volumetric estimate of the covering numbers 
N{K,D,t) (see e.g. jPj). Denote by \K\ the volume of K C M". 

Lemma 2.4. Let t > and let K,D cM."" be convex symmetric bodies. 
IftD C then 

S"l K\ 

N{K,D,t)<^. 

3. HALASZ TYPE LEMMA. 

Let ^i, . . . be independent centered random variables. To obtain 
the small ball probability estimates below, we have to bound the prob- 
ability that J2]=i is concentrated in a small interval. One standard 
method of obtaining such bounds is based on Berry-Esseen Theorem 
(see, e.g. |LPRTj ) . However, this method has certain limitations. In 
particular, if^j = tjSj, where tj G [1, 2] and ej are ±1 random variables, 
then Berry-Esseen Theorem does not "feel" the distribution of the co- 
efficients tj, and thus does not yield bounds better than c/ y/n for the 
small ball probability. To obtain better bounds we use the approach 
developed by Halasz [Halt IHa2j . 

Lemma 3.1. Let c > 0, < A < a/(27r) and let ^i,...,^n be in- 
dependent random variables such that E^j = 0, P (^j > a) > c and 
P (^i < -a) > c. Fory eR set 

n 

SA{y) = 5^ P (e, - e[y-nA,y + ttA]), 



where C,j is an independent copy of C,j ■ Then for any v G 
P 



<A1 / Sl{y)dy + ce- 



w 



5/2A 



3a/2 
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Proof. For t G M define 
and set 



k=l 



k=l 



Then by a Lemma of Esseen jEj , for any v G 



g = p 



< A < c 



■7r/2,7r/2] 



|y.(t/A)Mt 



Let be an independent copy of and let Vk = ik — i'k- Then 
P (|z/fc| > 2a) > 2c2 = c. We have 



(3.1) 
and 



|V9fc(t)P = Ecosz/fct 



Define a new random variable by conditioning on > 2a. For a 
Borel set A C M put 



P (Tfe G A) 



P (z/fc G A \ [-2a, 2a]) 



P (kfcl > 2a) 
Then by (jSH), 

1 - |(^fc(t)P > E(l -cosTfct) -P (|z/fc| > 2a) > c-E(l -cosr^t), 



so 



where 



|^(t)| <exp(-c7(t)), 



/(t) =E^(1 -COSTfct). 

fc=i 

Let T(m,r) = {t | /(t/A) < m, |t| < r} and let 



M = max fit/ A). 
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To estimate M from below, notice that 

M = max fit/ A) > - E > (1 — cos(rfe/A)t) dt 



2 sin(rfe/A)7r/2\ 



fc=i 



since |rfc|/A > 2a/A > 47r. 

To estimate the measure of T(m, vr/2) we use the argument of |Halj . 
For reader's convenience we present a complete proof. 

Lemma 3.2. Let < m< M/4. Then 

|T(m,7r/2)|<cJ--|T(M/4,7r)|. 



Proof. Let / = ^JM/Am. Taking the integer part if necessary, we may 
assume that / is an integer. For G N set 

k 

Sk = {Y.t,\t,enm,7:/2)}. 
i=i 

Note that 5*1 = T{m, tt/2). Since 

1 — cos a = 2sin^(a/2) 

and 



fc \ / k ^2 



sin^ 



o=i / \j=i / i=i 

we conclude that 5*^ C T{k'^m, krc /2). For k < I we have k^m < M, 
so (-7r/2, 7i/2) \ T{k'^m, kn/2) ^ 0. For a Borel set A denote yu(A) = 
\A n [— TT, 7r]|. Now we shall prove by induction that for all k < I 

fi{Sk)>{k/2)-fi{Si). 

Obviously, /i(5'2) = 15*21 > 2 • l^il, so this inequality holds for k = 2. 
Assume that fi{Sk^i) > {k — \)/2 ■ fi{Si). Note that the sets 5*^ are 
closed. Let v G (— 7r/2,7r/2) be a boundary point of 5*^. Such point 
exists since (— 7r/2,7r/2) \ 5*^ 7^ 0. Let {fj}°li be a sequence of points 
in (— 7r/2, 7r/2) \ Sk converging to v. Then (vj — Si) n Sk-i = 0, so by 
continuity we have 

fi{{v - Si) n Sk-i) = 0. 

Since the set 5*1 is symmetric, this implies 

/x{{v + 5*1) U Sk-i) = niv + Si) + niSk-i). 
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Both sets in the right hand side are contained in Sk+i (to see it for Sk^i 
note that G 5*2). Since v + Si C. [— 7r,7r], the induction hypothesis 
imphes 

k — 1 k -\- 1 

fJ'{Sk+i) > fj^{v + Si) + n{Sk-i) > fiiSi) H —-f^iSi) = -^—■fJ'iSi). 

Finally, since Si fl [— tt, tt] C T{Pm, vr), we get 

|T(/V^)|>^-|rKvr/2)|. 

□ 

We continue to prove Lemma (3. 11 Since 

Q<C I |<^(t/A)| dt<C I exp(-c7(t/A)) dt 

i[-7r/2,7r/2] J[-7r/2,7r/2] 
nn 

<C |T(m,7r/2)|e-^'™dm, 



Lemma 13.21 implies 

(3.2) g < • |T(^,vr)| +ce-^'^A^ < ■ |T(^, vr)| + ce-'^ 



Here for m > M/4 we used a trivial estimate |T(m, vr/2)| < vr. 

To complete the proof we have to estimate the measure of T = 
T(M/4, vr) from above. For any t G T we have 

n 

g{t) = ^Ecos(rfct/A) >n- M/4 > n/2. 

A:=l 

Let w(a;) = (1 — |a;|/7r) ■ X[-TT,n]{x) and put W = w. Then > and 
W{t) > c for |t| < TT. Hence by Parceval's equality, 

1^1 ^ (i) Vt - ^ (i) "I 

E5^^i;(rfe/A-|/) 

k=l 



dy. 

Since w < xi-TT,Tr], the last expression does not exceed 

Y 

7i,y + 7i]) dy 



/ f VP(rfc G [z-7rA,z + 7rA]) I dz. 
\k=i J 
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Since ^ [—2a, 2a] and ttA < a/2, we can integrate only over M \ 
[-3a/2,3a/2]. 
Substituting this estimate into ()3.2|) . we get 

Q<-Stt! I y]P(rfc G [2;-7rA,2; + 7rA]) I dz + ce-''''. 

To finish the proof, recall that the variables are symmetric. This al- 
lows to change the integration set in the previous inequality to (3a/2, oo). 
Moreover, if z G (3a/2, oo), then 

P (Tfc e[z- ttA, z + ttA]) <^-F{ukE[z- ttA, z + ttA]), 
so the random variables Tk can be replaced by Vk = ik — C,'k- 

□ 

Remark 3.3. A more delicate analysis shows that the term ce~^'" in 
the formulation of Lemma f3. II can always be eliminated. However, we 
shall not prove it since this term does not affect the results below. 

We shall apply Lemma 13.11 to weighted copies of the same random 
variable. To formulate the result we have to introduce a new notion. 

Definition 3.4. Let x G M™. For A > define the A-profile of the 

vector X as a sequence {Pk{x, A)}'^^-^ such that 

Pfc(x,A) = |{j I \xj\ G (A;A,(A; + 1)A]}. 

Theorem 3.5. Let (3 he a random variable such that E/5 = and 
P(/3 > c) > c', P(/3 < -c) > d for some c,d > 0. Let Pi . . . pm be 
independent copies of p. Let A > and let {xi . . . Xm) G be a vector 
such a < \xj\ < Cfeia for some a > 0. Then for any A < a/{2n) and 
for any v G M 



P 



jjXj 



fc=i 



Eft 

Proof. We shall apply Lemma f3. II to the random variables = XjPj. 

Let Ai (M) be the set of all probability measures on M. Consider the 
function F : A^(M) ^ M+ defined by 

POO 

Fif^)= / Sliy)dy, 

J 3a/ 2 
m ^ 

SA{y) = J^f'^ri • [1/ - ttA, y + ttA]). 

j=i l^il 



where 
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Since F is a convex function on A1(R), it attains the maximal value 
at an extreme point of this set, i.e. at some delta-measure 5t-, t G M. 
Note that in this case 

m 

SA{y) = \{j I t\xj\ G [y-7rA,y + 7rA]} = ^x{t\xj\ -y), 

where x = X[— ttA.ttA] is the indicator function of [— vrA, ttA]. For t < ^ 
we have t\xj\ < a/2, so S/\{y) = for any y > 3a/2, and thus F{6t) = 0. 
If t > then 

m m „oo 

Pi^t) = J2Yl / X{t\xj\-y)x{t\xi\-y)dy 



j=i 1=1 



3a/2 



<27rA|{(j,/) I t||a;,| - < nA}\=g{t). 
Since the function g is decreasing, 



^ oo 

1=1 



\xj\ — I A 



< 2nA ■ C}p 



<CAj2\{j\\x,\e{kA,{k + l)A]}\\ 



k=l 



The last inequality holds since we can cover each interval [/A— 27rAC, lA- 
271 AC] by at most 27rC + 2 intervals (A;A, {k + 1)A]. 

Let n be the distribution of the random variable jS—jS', where is an 
independent copy of /3. Applying Lemma (3.11 to the random variables 

= Xj ■ Pj, we have 



P 



C 



< A < 



C 



m 



5/2A 



+ ce" 



^^2!! \i3 I l^.l e {kA, {k + l)A]}p + ce-'^ 



k=l 



Since the sum in the right hand side is at least m, the second term is 
negligible compare to the first one. Thus, 



P 



2C' 



k=l 



□ 
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4. Small ball probability estimates. 

Let G be an n X n Gaussian matrix. If x G S*""^ is any unit vector, 
then y = Gx is the standard Gaussian vector in R". Hence for any 
t > we have P < t) < t ■ \/2j7T for any coordinate. Moreover, 

P(||Z/|| <t-V^)< (27r)-"/\ol(tv^52") < (Ct)". 

We would hke to have the same small ball probability estimates for the 
random vector y = Ax. However, it is easy to see that it is impossible 
to achieve such estimate for all directions x G 5*"^^. Indeed, if A is a 
random ±1 matrix and x = (l/-\/2, l/"\/2, . . . 0), then P = 0) = 
1/2 and P (y = 0) = 2"". Analyzing this example, we see that the 
reason that the small ball estimate fails is the concentration of the 
Euclidean norm of x on a few coordinates. If the vector x is "spread" , 
we can expect a more regular behavior of the small ball probability. 

Although we cannot prove the Gaussian type estimates for all di- 
rections and all t > 0, it is possible to obtain such estimates for most 
directions provided that t is sufficiently large [t > to). Moreover, the 
more we assume about the regularity of distribution of the coordinates 
of X, the smaller value of to we can take. This general statement is 
illustrated by the series of results below. 

The first result is valid for any direction. The following Lemma is a 
particular case of jLPRTj . Proposition 3.4. 

Lemma 4.1. Let A be an n x n matrix with i.i.d. suhgaussian entries. 
Then for every x G 5'"""'^ 

^{\\Ax\\ < %TIv^) < exp(-qnF). 

The example considered at the beginning of this section shows that 
this estimate cannot be improved for a general random matrix. 

If we assume that all coordinates of the vector x are comparable, then 
we have the following Lemma, which is a particular case of Proposition 
3.4 jLPl^TV2j (see also Proposition 3.2 [EFEEl). 

Lemma 4.2. Let (3 he a random variable such that E/3 = 0, = 1 
and let (3i, . . . , f3m be independent copies of (3. Let < r < R and let 
Xi, . . . ,Xm G M 6e such that rj^fm < \xj\ < R/y/m for any j. Then 
for any t > ^Jm and for any w G M 



Here qpi and Qpi depend only on r and R. 
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Proof. The proof is based on Berry- Esseen theorem (cf., e.g., Sec- 
tion 2.1). 

Theorem 4.3. Let (Cj)i!Li be a sequence of independent random vari- 
ables with expectation and finite third moments, and let := ^2^=1 ^ICjP- 
Then for every r G M one has 



P 



(J^Q <TA)-r{g< r)| < {c/A')J2e\Q\\ 

where g is a Gaussian random variable with N{0, 1) distribution and 
c> 1 is a universal constant. 



Let Cj = (3jXj. ThenA^ 



^ > r^. Since the random 



variables Pj are copies of a subgaussian random variable P, E|/?|^ < C 



for some absolute constant C. Hence, EYlT=i ICjP < 



C'/ \fm. By Theorem 14. HI we get 

m 

Eft 



< 



P 



< t < P 



< C"t + 



- <^ < 



provided t > 
If X = (1/ 



c" 



□ 



1/ 



then 



P 



>c/ 



m. 



This shows that the bound t > ^Jm in Lemma 14.21 is necessary. 

The proofs of Lemma l^^Tl and Lemma l^^ are based on Paley-Zygmund 
inequality and Berry-Esseen Theorem respectively. To obtain the lin- 
ear decay of small ball probability for t < qi^j/ \/rn, we use the third 
technique, namely Halasz method. However, since the formulation of 
the result requires several technical assumptions on the vector x, we 
postpone it to Sectional where these assumptions appear. 

To translate the small ball probability estimate for a single coordi- 
nate to a similar estimate for the norm we use the Laplace transform 
technique, developed in |LPRTj . The following Lemma improves the 
argument used in the proof of Theorem 3.1 |LPRTj . 

Lemma 4.4. Let A > and let Y be a random variable such that for 
anyv eM and for any t > A, F{\Y-v\ > t) < Lt. Lety = 
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be a random vector, whose coordinates are independent copies of Y . 
Then for any z e M" 

P {\\y-z\\<Ay/^) < (CfeiLA)". 

Proof. We have 



P {\\y - ^11 < A^) = P J2{Yi - z,f < Ar 

\i=i 

( 1 " 

P U--5^(r,-z,;)^>o 

<Eexp [n-^^yy^-z,)^ 

n 

= e'^- J]Eexp(--(r,-^,)2). 

i=l 

To estimate the last expectation we use Lemma IHTTl 

Eexp(-l(Fi - Zif) = j^F (^exp ( - ^(Y, - z,f'^ > ds 



2Me"" F{\Yi - Zi\ < Au)du 

< I ue-'^'/^LAdu 
Jo 

/oo 
ue-'^'/^LAu du 

< CLA. 

Substituting this into the previous inequahty, we get 
P {\\y - z\\ < Av^) < (e ■ CLA)". 

5. Partition of the sphere. 



□ 



To apply the small ball probability estimates proved in the previous 
section we have to decompose the sphere into different regions depend- 
ing on the distribution of the coordinates of a point. We start by 
decomposing the sphere S""^ in two parts following |LPRTt FLPRTVII 
ILPRTV2] . We shall define two sets: Vp - the set of vectors, whose 
Euclidean norm is concentrated on a few coordinates, and Vs - the set 
of vectors whose coordinates are evenly spread. Let r < 1 < R he 
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the numbers to be chosen later. Given ) e set 

a{x) = {i I \xi\ < R/^Jn}. Let Pi be the coordinate projection on the 
set / C {,..., n}. Set 

Vp = {xeS''-^ I <r} 

r5 = {xGS"-i I >r}. 

First we shall show that with high probability \\Ax\\ > C^fn for any 
X e Vp. 

For a single vector x G this probability was estimated in Lemma 
14. 1[ We shall combine this estimate with an e-net argument. 

Lemma 5.1. For any r < 1/2 

logiV(\/p,i?2",2r)<^-log {^-^y 

Proof. If X G 5 then |{1, . . . , n} \ a{x)\ < n/R. Hence, the set Vp is 
contained in the sum of two sets: rBl]^ and 

Wp = {xeB^\ |supp(x)| < n/R^}. 

Since Wp is contained in the union of unit balls in all coordinate sub- 
spaces of dimension I = n/R, Lemma f2.4l implies 

N{Wp,Blr) < Q ■NiB^„Bl,r) < Q -0)'. 

Finally, 

□ 

Recall that C fcn < Qr^ Set r = Cg3j/2Cj^ and choose the number 
i? > 1 so that 

R \ r J 2 
For these parameters we prove that the norm of Ax is bounded below 
for all X & Vp with high probability. 

Lemma 5.2. 

P (3x G Vp I \\Ax\\ < (%Tlv^/2) < 2exp(-qnp). 

Proof. By Lemma lOl the set Vp contains a (Qfam/ 2Cfe:3j)-net Af in the 
£2-nietric of cardinality at most exp(qjip/2). Let 

Qo = {u\ \\A\\ > Cfejv^} 




INVERTIBILITY OF RANDOM MATRICES; NORM OF THE INVERSE 15 



and let 

= {uj \ 3x E M \\A{uj)x\\ < QraJn}. 

Then Lemma (4. II implies 

P(fio) +P(J^p) < exp(-?2) +exp(-qTTF) < 2exp(-(g3p). 

Let uj ^ Qp. Pick any x G Vp. There exists y E M such that 
ll^; - y\\2 ^ %Il/2<HO Hence 

\\Ax\\ > \\Ay\\ - \\Aix -y)\\> %t1v^ - \\A : Bl^ ^ B^\\ ■ \\x - y\\. 



□ 



For a; = {xi, . . . , x„) G denote 

r 



(5.1) j(x) = <!j|-^<ki< ^ 



Note that 



so 



|J(x)| > {r^/2R^) ■n=:m. 

Let < A < r I2\fn be a number to be chosen later. We shall cover 
the interval {■^, ^] by 

'i?-r/2" 



k 



consecutive intervals (jA, (j + l)A], where j = fco, (^o + l), • • • , (^0 + ^), 
and fco is the largest number such that /cqA < r /2y/n. Then we shall 
decompose the set Vs in two subsets: one containing the points whose 
coordinates are concentrated in a few such intervals, and the other 
containing points with evenly spread coordinates. This will be done 
using the A-profile, defined in 13.41 Note that if m coordinates of the 
vector X are evenly spread among k intervals, then 



2 

m 



J2P^ix,A)r.'^r^m'/-A. 

i=l 

This observation leads to the following 
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Definition 5.3. Let A > and let Q > 1. We say that a vector 
X E Vs has a (A, Q)-regular profile if there exists a set J C J{x) such 
that I J| > m/2 and 



Here a;|j G M"' is a vector with coordinates x\j{j) = x{j) ■ XjU)- 

If such set J does not exist, we call x a vector of (A, (5)-singular 
profile. 

Note that T^Zi^ii^lj^ ^) ^ Hence, if A < then 

every vector in Vs will be a vector of a (A, (5)-singular profile. 

Vectors of regular and singular profile will be treated differently. 
Namely, in Section IHl we prove that vectors of regular profile satisfy the 
small ball probability estimate of the type Ct for t > A. This allows 
to use conditioning to estimate the probability that \\Ax\\ is small for 
some vector x of regular profile. In Section [7| we prove that the set 
of vectors of singular profile admits a small e-net. This fact combined 
with Lemma (4.21 allows to estimate the probability that there exists a 
vector X of singular profile such that \\Ax\\ is small using the standard 
e-net argument. 



To estimate the small ball probability for a vector of a regular profile 
we apply Theorem 13.51 

Lemma 6.1. Let A < ^J^ - Let x E Vs be a vector of {A, Q) -regular 
profile. Then for any t > A 



Proof. Let J C {1, . . . , n}, \J\ > m/2 be the set from Definition 15.31 
Denote by Ejc the expectation with respect to the random variables 
Pj, where j G J'^ = {1, . . . , n} \ J. Then 



oo 




1=1 



6. Vectors of a regular profile. 




Hence, it is enough to estimate the conditional probability. 
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Recall that /? is a centered subgaussian random variable of variance 
1. It is well-known that such variable satisfies P (/3 > c) > c', F {j3 < 
— c) > c' for some absolute constants c, c'. Moreover, a simple Paley- 
Zygmund type argument shows that this estimates hold if we assume 
only that Kf3 = and the second and the fourth moment of f3 are 
comparable. Hence, for t = A the Lemma follows from Theorem 13 .51 
where we set a = r/y/n, Cfei= R/r. 

To prove the Lemma for other values of t, assume first that t = As = 
2^ A < for some s G N. Consider the A^-profile of x\j: 

Pi{x\j,As) = \{j e J I \xj\ e (lAs, {I + i)A,]}|. 

Notice that each interval (/A^, (/ + l)As] is a union of 2* intervals 
(iA, (i + 1)A]. Hence 

oo oo 

5^Pf(x|j, A,) < VY.P^^''\J^^) ^ TQrrt'l^A = Qm'/H. 



1=1 



1=1 



Applying Theorem 13 . 51 with A replaced by A^ and v' = v + J^j^J'^ Pj^j^ 
we obtain 



P 



< 1 1 f3j, jer]< C^t. 



For 2'*A < t < 2*+^A the result follows from the previous inequality 
applied for t = 2*A. If t > (^^/m = ^^y , Lema lOl implies 



P 



Finally, if ^ < t < the previous inequality applied to to 

^I P P implies 



P 



'3^3 > 



< t I f3j, J G jM < q^to < CQt, 



where C = q^- ^^^f^ ■ ^. □ 

Now we estimate the probability that ||y4(ci;)2;|| is small for some 
vector of a regular profile. 

Theorem 6.2. Let A > and let U be the set of vectors of {A,Q)- 
regular profile. Then 



P 3x G f/ I \\Ax\\ < 



A 
27^ 



< C^An. 
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Let Q be the event described in Theorem 16.21 Denote the rows of A 

for any u E Q there 



-i\r| 



by ai, . . . , a„. Note that since \\A ^\\ = 
exists a vector u = {ui, . . . , Un) € 5"^^ such that 

uioi + . . . + u„a„ = z, 

where ||2;|| < s. Then Vl = U'^^iQk, where Qk is the event \uk\ > \fn. 
Since the events fifc have the same probabihty, it is enough to estimate 

To this end we condition on the first n — 1 rows of the matrix A = 
A{u): 

P (n„) = E„,,...,,„_,P (n„ I ai, . . . , a„_i). 

Here '&ai,...,an-i is the expectation with respect to the first n — 1 rows 
of the matrix A. Take any vector y E U such that 

n-l 

i=i 

If such vector does not exist, then \\Ay\\ > s for all y E U, and so 
LO ^ fl. Note that the vector y can be chosen using only ai, . . . , a„_i. 
We have 



a„ = — (miOi + . . . + Mn_ian_i - z) 

Ur,, 



so for uj E fir 



\{an,y)\ 



Ur, 



n-l 



i=i 

n-l \ /n-l 



1/2 



o=i 



o=i 



The row a„ is independent of ai, . . . , a„_i. Hence, Lemma IHTTl implies 

F(fin) <F (|(a„,|/)| < A I ai,...,a„_i) 



i=i 



< A I ai, . . . , a„_i < Qn^A, 



and so P (r^) < (%TlQAn. 



□ 
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7. Vectors of a singular profile. 



We prove first that the set of vectors of singular profile admits a 
small A-net in the ^t^D-metric. 



Lemma 7.1. Let Qmra'^/^ < A < n where Ctn\= ^ (^nd let Ws 
be the set of vectors of (A, Q) -singular profile. Let r] > be such that 



where C{r]) is the function defined in Lemma \2.1\ Then there exists a 
A-net M in Ws in loo-metric such that 



Remark 7.2. Lemma |23] implies that there exists a A-net for S"""^ in 
the £oo-nietric with less than (CAZ-y/n)" points. Thus, considering only 
vectors of a singular profile, we gain the factor y^ini"- in the estimate 
of the size of a A-net. 

Proof. Let J C {1, . . . , n} and denote J' = {1, . . . ,n} \ J. Let Wj C 
Ws be the set of all vectors x of a (A, Q)-singular profile for which 
J{x) = J. We shall construct A-nets in each Wj separately. To this 
end we shall use Lemma l2.ll to construct a A-net for the set PjWj, 
where Pj is the coordinate projection on R'^. Then the product of this 
A-net and a A-net for the ball B2 will form a A-net for the whole Wj. 

Assume that J = {1, . . . , /}, where / > m. Let Ji, . . . , be consecu- 
tive subintervals {iA, {i + 1)A], i = k^, . . . , kQ-\- k, covering the interval 
Iy/^, which appear in the definition of profile. Recall that 



The restriction on A implies that k < m. Let di be the center of the 
interval Jj. Set 



Then = (2/c)'. Let A/j be the set of all x G Aij for which there 

exists a vector y G Wj such that —A/2 < yj — xj < A/2 for all j G J. 
The set Afj forms a A-net for Wj in the i^o metric. To estimate its 
cardinality we use the probabilistic method. 

Let X{1), . . . ,X{1) be independent random variables uniformly dis- 
tributed on the set {1, . . . , k}. Let C {1, . . . , fc}' be the set of all 
/-tuples (f (1), . . . , f (/)) such that \xj\ = dy^^, j = 1,...,/ for some 



Civ) < qfeiQ, 




k 



R-r/2 
VnA 



Mj = {x G M I \xj\ G {di, . . .,dk} for j G J}. 
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X = (xi, . . . ,xi) G Afj. Since both J^j and Afj are invariant under 
changes of signs of the coordinates, 

P((X(l),...,X(/))GiV) = ^. 

Let {X{1), . . . ,X{1)) G and let x G be such that Xj = dx(j)- Let 
y G Wj be a vector such that —A/2 < yj — Xj < A/2 for all j G J. 
Then for any j G J, yj G /j implies that X{j) = i. Let C J be any 
set containing at least m/2 elements. Then 

oo fc 

Y,P-{y\E,A) = J2\{j^E\X{j)=z}\'. 

i=l i=l 

Since ?/ is a vector of a singular profile, this implies 

\{3 e E I X(j) = z}p > gm^/^A = Cfej- > C(r^) ■ ^. 



Now Lemma (2. II implies that P (A^) < r^', so 

To estimate the cardinality of the A-net for the whole Wj we use 
Lemma in Since A < 1/v^, C B^, so 

N{PjWj,Bi,/\) < N{Bi\Bi,/X) < 3"-'^7^ < (^^/^' 

Since the function f{t) = {a/ty is increasing for < t < a/e, the 
right-hand side of the previous inequality is bounded by {c/A^/n)"'. 
Hence, 

NiWj, 5^, A) < NiPjWj, Bi,A)- NiPjWj, 5^', A) 
Finally, set 

^f= U ^fJ. 

\J\>m 

Then 

l=m\J\=l V V / 



Thus, Lemma mi holds with q7m= m/n = j^. □ 
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Now we are ready to show that \\Ax\\ > c for all vectors of a (A, Q)- 
singular profile with probability exponentially close to 1. 

Theorem 7.3. There exists an absolute constant Qo with the following 
property. Let A > QtWI'~^^^, where Cjo = niaxfq^ Qtht)- Denote by 
Q/^ the event that there exists a vector x G Vs of {A, Qo) -singular profile 
such that \\Ax\ 



<fn. 



Then 
F{Qa) < 3exp( 



-n] 



Proof. We consider two cases. First, we assume that A > Ai = qoj/n. 
In this case we estimate the small ball probability using Lemma 14.21 
and the size of the e-net using Lemma (7. II Note that only the second 
estimate uses the the profile of the vectors. Then we conclude the proof 
with the standard approximation argument. 

The case A < Ai is more involved. From Case 1 we know that 
there exists Qi such that all vectors of (Ai, (5i)-singular profile satisfy 
\\A-x\\ > with probability at least 1 — e~". Hence, it is enough 
to consider only vectors whose profile is regular on the scale Ai and 
singular on the scale A. For these vectors we use the regular profile in 
Lemma mm to estimate the small ball probability and singular profile in 
Lemma f7. II to estimate the size of the e-net. The same approximation 
argument finishes the proof. 

Case 1. Assume first that A > Ai = qo/ra. Let Qi > 1 be a 
number to be chosen later. Let Ai be the smallest 7;7=i-net in the set 
of the vectors of (A, (5i)-singular profile in i^o metric. 

Let X & Vs and let J = J{x) defined in flS.lj) . Denote J'^ = 
{1, . . . ,n}\ J. Then Lemma [4.21 implies 



P 



<t\ = E,cP 



for all t > qoi/ y^n. Since A^/n > qpi/ y^n, by Lemma [4.41 we have 

P {\\Ax\\ < An) < (%^v^r 



and so, 
(7.1) 



P (3a; G I \\Ax\\ < An) < \M\{q^^/Ey 



We shall show that Qi can be chosen so that the last quantity will be 
less than (2e)~". Recall that by Lemma l7.H there exists a A-net J\f 
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for the set of vectors of (A, (5i)-singular profile satisfying 
provided 

(7.2) Civ) < q^i. 

Covering each cube of size A with the center in JV by the cubes of size 
and using Lemma 1231 we obtain 

1^1 < I ATI . iV(AS^, ABl, ^) < ( ^^^ ^'^) " • 
Substitution of this estimate into ()7.1|1 yields 

F{3xeAf\ \\Ax\\ < An) < (^^^j^^"^) " (CfaAv^)" 

< (C'r/ini)'^. 

Now choose rj so that C'rj'^ < 1/e and choose Qi satisfying ()7.2p . With 
this choice the probability above is smaller than e'". Combining this 
estimate with LemmaEISl we have that \\A\\ < Qr^tJ^ and \\Ax\\ > An 
for all X G A/" with probability at least 1 — 2e~". 

Let y E Vs he a. vector of (A, (5i)-singular profile. Choose x E J\f 
such that ||x - y\\^ < Then \\x - y\\ < and 

\\Ay\\ > \\Ax\\ - \\A{x -y)\\ > An - \\A\\ \\x - y\\ > ^n. 

Case 2 Assume that CtntT-'^^^ < A < Ai = qoi/r?-. Let f2i be the 
event that \\Ax\\ < ^n = qoi/2 for some vector of (Ai, (5i)-singular 
profile. We proved in Case 1 that 

(7.3) P(fii)<2e-'^. 

Let Q2 > 1 be a number to be chosen later and let W be the set of 
all vectors of (Ai, (5i)-singular and (A, Q2)-regular profile. By Lemma 
16.11 any vector x E W satisfies 



P 



for all t > Ai. 

Now we can finish the proof as in Case 1. Since Ay^ > Ai, Lemma 
14.41 implies 

P {\\Ax\\ < An) < (C'Av^)" 
for any x G W. Here C = Cjo|- QqQi. 
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Let M be the smallest g^^-net in W in £00 nietric. Note that A > 
^ Arguing as in the Case 1, we show that 

lA^I < ( 

for any rj satisfying 

(7.4) C{r^) < G^2. 

Hence, 

P (3x G A/" I \\Ax\\ < An) < |7V|(C'Av^)" < (C'r^lHI)". 

Choose 7] so that the last quantity is less than e~" and choose Q2 so 
that ()7.4j) holds. Then the approximation argument used in Case 1 
shows that the inequality 

Uy\\ > 

holds for any y &W with probability greater than 1 — e~". Combining 
it with ()7.3|) . we complete the prof of Case 2. Finally, we unite two 
cases setting Qq = max(Qi, Q2). □ 

8. Proof of Theorem II. 1L 

To prove Theorem 11.11 we combine the probability estimates of the 
previous sections. Let e > qiTi / y/n, where the constant qm will be 
chosen later. Define the exceptional sets: 

r^o = {^^ I ll^ll > QmJn}, 

Qp = {u \3x EVp \\Ax\\ < Qo\/ri }. 

Then Lemma 12.31 and Lemma 15.21 imply 

P(f]o) +P(fip) < 3exp(-q|3^). 
Let Qo be the number defined in Theorem 17.31 Set 

A = . 

The assumption on e implies A > Qtw-'^^'^ if we set go = 2CfemQo • 
Qt^} Denote by Ws the set of vectors of (A, (5o)-singular profile and 
by Wr the set of vectors of (A, Qo)-i'egular profile. Set 

ns = {uj\3xeWs \\Ax\\ <^n= 

nn = {uj \3x eWr \\Ax\\ < = 777^£ ■ «"^^'}- 
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By Theorem Ea ^ (^s) < 36"", and by Theorem lOl P (fi/?) < e/2. 
Since 5""^ = VpUWsUWR,we conclude that 

F{uj\3xe S''-^ \\Ax\\ < e ■ n'^/^} < e/2 + 4exp(-qop) < e 

for large n. □ 

Remark 8.1. The proof shows that the set of vectors of a regular 
profile is critical. On the other sets the norm of Ax is much greater 
with probability exponentially close to 1. 
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